Face authentication system and face authentication method

ABSTRACT

A face authentication system including a storage to store a face image of each of a plurality of persons who are permitted to enter in advance and information on a zone; an image processing unit to detect a face of at least one person in an image captured by at least one camera having identification information, and generate a face cut-out image of the at least one person; a zone determination unit to determine a zone where a person in the face cut-out image is captured; a collation unit to collate the face cut-out image with the face image of each of the plurality of persons; and an output control unit to, when it is determined that there is not a face of a person appearing in the face cut-out image, superimpose a detection frame indicating a face of a person who is not permitted to enter the zone.

BACKGROUND OF THE INVENTION 1. Field of the Invention

The present disclosure relates to a face authentication system and aface authentication method.

2. Background Art

JP-A-2018-92293 discloses an entrance/exit management system thatmanages entrance/exit of a user in an entrance/exit gate of a managedarea. The entrance/exit management system includes: a face imagedatabase where face image data of a user and data related to the userare registered in association with each other; a plurality of isolationwalls consisting of a plurality of gates having a width that allows asingle user to pass therethrough; an imaging unit for acquiring imagedata of the plurality of gates; a face authentication unit forperforming face authentication of a plurality of persons acquired in animage acquisition zone in the registered face image data and the imagedata acquired by the imaging unit; and a notification unit for notifyingthe user passing through the gate of the face authentication result.

SUMMARY OF THE INVENTION

In a configuration of the entrance/exit management system ofJP-A-2018-92293 described above, it is possible to capture an image ofthe single user passing through the image acquisition zone set accordingto the width of the gate through which the user can pass, andauthenticate whether the user is a user registered in the face imagedatabase based on the acquired image data of the user. However, theentrance/exit management system does not consider the passage of each ofthe plurality of users, and when the plurality of users pass through thegate at the same time, there is a possibility that the faceauthentication of the user cannot be accurately executed since theplurality of users are showed in the image data acquired in one imageacquisition zone.

The present disclosure has been made in view of the above circumstances,and an object thereof is to provide a face authentication system and aface authentication method that detect and output a person who does nothave permission to enter among a plurality of persons who enter amonitored area and more efficiently support a monitoring work of a user.

The present disclosure provides a face authentication system including astorage configured to store a face image of each of a plurality ofpersons who are permitted to enter in advance and information on a zonein association with each other, an image processing unit configured todetect a face of at least one person in an image captured by at leastone camera having identification information, and generate a facecut-out image obtained by cutting out a detected face portion of the atleast one person, a zone determination unit configured to determine azone where a person in the face cut-out image is captured based on theidentification information of the camera, a collation unit configured tocollate the face cut-out image with the face image of each of theplurality of persons who are permitted to enter the zone in advance, andan output control unit configured to, when it is determined by collationthat there is not a face of a person appearing in the face cut-out imageamong the face images of the plurality of persons who are permitted toenter the zone in advance, superimpose a detection frame indicating aface of a person who is not permitted to enter the zone on the facecut-out image, and output the face cut-out image with the detectionframe.

In addition, the present disclosure provides a face authenticationmethod including storing a face image of each of a plurality of personswho are permitted to enter in advance and information on a zone inassociation with each other into a storage, detecting a face of at leastone person in an image captured by at least one camera havingidentification information, and generating a face cut-out image obtainedby cutting out a detected face portion of the at least one person,determining a zone where a person in the face cut-out image is capturedbased on the identification information of the camera, collating theface cut-out image with the face image of each of the plurality ofpersons who are permitted to enter the zone in advance, and when it isdetermined by collation that there is not a face of a person appearingin the face cut-out image among the face images of the plurality ofpersons who are permitted to enter the zone in advance, superimposing adetection frame indicating a face of a person who is not permitted toenter the zone on the face cut-out image, and outputting the facecut-out image with the detection frame.

According to the present disclosure, it is possible to provide the faceauthentication system and the face authentication method that detect andoutput a person who does not have permission to enter among a pluralityof persons who enter the monitored area, and more efficiently support amonitoring work of a user.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a view showing an overall configuration example of a faceauthentication system according to a first embodiment.

FIG. 2 is a view showing an internal configuration example of the faceauthentication system according to the first embodiment.

FIG. 3 is a view showing an example of a registrant memory table.

FIG. 4 is a sequence diagram showing an operation procedure example ofthe face authentication system according to the first embodiment.

FIG. 5 is a sequence diagram showing an operation procedure example ofthe face authentication system according to the first embodiment.

FIG. 6A is a view illustrating an example of alert notification.

FIG. 6B is a view illustrating an example of the alert notification.

FIG. 6C is a view illustrating an example of the alert notification.

DETAILED DESCRIPTION OF THE EXEMPLARY EMBODIMENT

Hereinafter, an embodiment of a face authentication system and a faceauthentication method according to the present disclosure will bedescribed in detail with reference to the accompanying drawings.However, unnecessarily detailed description may be omitted. For example,a detailed description of a well-known matter or a repeated descriptionof substantially the same configuration may be omitted. This is to avoidunnecessary redundancy in the following description and to facilitateunderstanding of those skilled in the art. It should be noted that theaccompanying drawings and the following description are provided forthose skilled in the art to fully understand the present disclosure, andare not intended to limit the claimed subject matter.

An overall configuration example of a face authentication system 100according to a first embodiment will be described with reference to FIG.1 . FIG. 1 is a view showing a configuration example of the faceauthentication system 100 according to the first embodiment.

The face authentication system 100 according to the first embodiment isprovided in a building such as a company, a condominium, or a commercialfacility. The face authentication system 100 captures images of one ormore persons who enter a zone in a building (for example, an area suchas a doorway, a room, a floor, or an elevator of the building),determines whether the captured person is permitted to enter in theentered zone, and outputs the determined result to a user (for example,an administrator, or the like). The face authentication system 100includes cameras C1, C2, C3, . . . , a server S1 and a terminal deviceP1. Note that the server S1 and the terminal device P1 may be integrallyconfigured.

A plurality of cameras C1, . . . are connected to the server S1 so as tobe able to perform data communication, and transmits a captured image tothe server S1, respectively. The plurality of cameras C1, . . . areinstalled in a plurality of zones A, B, . . . serving as monitored areaswhere the entry of a plurality of persons is monitored, and capturesimages of the plurality of persons entering the plurality of zones A, B,. . . , respectively. For example, in the example shown in FIG. 1 , thecamera C1 is installed in the zone A. In addition, the plurality ofcameras C2, C3 are installed in the zone B.

Each of the plurality of cameras C1, . . . is assigned cameraidentification information that can specify and identify an installedzone and an installed position by a user operation. The identificationinformation of each of the plurality of cameras C1, . . . is recorded inthe server S1. Note that the identification information of each of theplurality of cameras C1, . . . may be recorded in the terminal device P1and transmitted to the server S1. Each of the plurality of cameras C1, .. . detects a face of a person or faces of the plurality of persons fromthe captured image, generates a cut-out face cut-out image, andtransmits the generated face cut-out image and the identificationinformation of the camera to the server S1. Note that, when each of theplurality of persons is shown in the captured image, the face cut-outimage cut out here is cut out and generated as a face cut-out imageincluding the face of each of the plurality of persons.

Each of the plurality of cameras C1, . . . generates position coordinateinformation of a face indicating a position coordinate of a detectedface of the person. Each of the plurality of cameras C1, . . . transmitsthe face cut-out image, the position coordinate information of the faceof the person shown in the face cut-out image, and the identificationinformation of the camera to the server S1.

A processing of detecting the face of the person from the capturedimage, a processing of acquiring the position coordinate information ofthe detected face of each person, and a processing of generating theface cut-out image may be executed by the server S1. In such a case, theserver S1 acquires the captured image captured by each of the pluralityof cameras C1, . . . and the identification information of the camerathat captures the captured image.

The terminal device P1 is connected to the server S1 so as to be able toperform data communication. The terminal device P1 displays alertnotification transmitted from the server S1 on a monitor. The terminaldevice P1 may record zone identification information (for example, aposition, a zone name, or the like) of each of the plurality of zones A,. . . for executing face authentication input by a user operation, andthe camera identification information of each of the plurality ofcameras C1, . . . (for example, management number, manufacturing number,or the like). In addition, the terminal device P1 may generate andrecord grouping information including identification information of thecamera installed for each zone input by a user operation. When theterminal device P1 generates the grouping information including theidentification information of the camera, the terminal device P1 maytransmit the recorded zone identification information, cameraidentification information, and grouping information to the server S1and record the information.

The terminal device P1 accepts a user operation regarding a face imageof a person who can enter any one of the plurality of zones A, . . . ,information on a zone where the entry is permitted, and an expirationdate indicating a deadline for the entry. The face image of the personmay be feature amount data of a face.

In the following description, an example in which the faceauthentication is executed by a collation processing using face imagesof a plurality of persons will be described, but the face authenticationmay be executed by the collation processing using a feature amount ofthe face.

The server S1 is connected to the plurality of cameras C1, . . . and theterminal device P1 so as to be able to perform data communication.

The server S1 generates and records grouping information including theidentification information of the camera installed for each zone inputby a user operation. The server S1 transmits the recorded zoneidentification information, camera identification information, andgrouping information to the terminal device P1. In addition, the serverS1 generates a registrant memory table TB1 in which the face image orthe feature amount data of the face of each of the plurality of personsregistered by the user, the information on a zone, and the expirationdate are associated with each person, and records (registers) aregistrant database 33 (see FIG. 2 ). The registrant memory table TB1may be transmitted from the server S1 to the terminal device P1 andrecorded in a registrant database 15. The server S1 records the zoneidentification information of each of the plurality of zones A, . . .transmitted from the terminal device P1, the camera identificationinformation of each of the plurality of cameras C1, . . . , the groupinginformation of the camera and the registrant memory table TB1.

Regarding the various setting information of each of the plurality ofpersons recorded in the registrant memory table TB1 or the registrantdatabase 15 (the face image, the information on the zone, and theexpiration date), the server S1 may delete various setting informationof a person whose expiration date has expired, or may continuouslyrecord various setting information without deleting the information.

The server S1 receives the face cut-out image and the cameraidentification information (for example, an IP address for each camera,or the like) from each of plurality of cameras C1, . . . . The server S1determines a zone in which the face cut-out image is captured based onthe camera identification information. The server S1 collates the faceimage of each of the plurality of persons permitted to enter in the zoneidentification information determined in the registrant memory table TB1with the face cut-out image.

When it is determined that the person shown in the face cut-out image ispermitted to enter the zone as a result of the collation, the server S1further determines whether the expiration date for which the person ispermitted to enter has expired.

The server S1 generates an alert notification (see FIGS. 6A and 6B) as adetermined result when the person shown in the face cut-out image is notpermitted to enter the zone or the expiration date for which the personis permitted to enter has expired. The generated alert notification istransmitted from the server S1 to the terminal device P1, and isdisplayed on a monitor 13 (see FIG. 2 ) in the terminal device P1.

The alert notification may be generated by the terminal device P1. Insuch a case, the terminal device P1 acquires the face cut-out imageincluding the person who is not permitted to enter the zone as thedetermined result from the server S1, and the position coordinateinformation of the face indicating the position of the face of theperson who is not permitted to enter in the face cut-out image, andgenerates the alert notification.

On the other hand, the server S1 generates an alert notification (seeFIG. 6A) as a determined result using the face image and the facecut-out image recorded (registered) in the registrant memory table TB1when the person shown in the face cut-out image is permitted to enterthe zone and the expiration date for which the person is permitted toenter has expired. The generated alert notification is transmitted fromthe server S1 to the terminal device P1, but is displayed on the monitor13 (see FIG. 2 ) in the terminal device P1.

FIG. 2 is a view showing an internal configuration example of the faceauthentication system 100 according to the first embodiment.

The terminal device P1 includes a communication unit 10, a processor 11,a memory 12, the monitor 13, an operation unit 14, and the registrantdatabase 15. The registrant database 15 shown in FIG. 2 may be realizedas an external terminal device or an external storage device connectedto the terminal device P1 so as to be able to perform externalcommunication. Although not shown in FIG. 2 , the terminal device P1 maybe connected to a face image database that records (registers) each ofthe face images of a plurality of persons as an external terminal deviceor an external storage device. In such a case, when generating theregistrant memory table TB1, the terminal device P1 acquires a faceimage of a person to be recorded (registered) from the face imagedatabase.

The communication unit 10 is connected to a communication unit 30 in theserver S1 so as to be able to perform data communication via a network.The communication unit 10 receives the determined result (alertnotification) as a face authentication result transmitted from theserver S1 and inputs the result to the processor 11.

The processor 11 as an example of an output control unit is configuredusing, for example, a central processing unit (CPU) or a fieldprogrammable gate array (FPGA), and performs various processes andcontrols in cooperation with the memory 12. Specifically, the processor11 refers to a program and data held in the memory 12, and executes theprogram to implement functions of the respective units.

When the determined result as the face authentication result input fromthe communication unit 10 transmitted from the server S1 is alertnotification, the processor 11 outputs and displays the alertnotification on the monitor 13. When the processor 11 acquires the facecut-out image including the person who is not permitted to enter thezone as the determined result from the server S1 and the positioncoordinate information of the face indicating the position of the faceof the person whose entry is not permitted in the face cut-out image,the processor 11 may superimpose a detection frame on the face cut-outimage based on the position coordinate information of the face togenerate the alert notification, and output and display the generatedalert notification on the monitor 13.

The memory 12 as an example of a storage includes, for example, a randomaccess memory (RAM) as a work memory used when executing each processingof the processor 11, and a read only memory (ROM) for storing programsand data defining an operation of the processor 11. The RAM temporarilystores data or information generated or acquired by the processor 11. Aprogram that defines the operation of the processor 11 is written in theROM. The memory 12 may record the registrant memory table TB1transmitted from the server S1.

The monitor 13 is configured using a display such as a liquid crystaldisplay (LCD) or an organic electroluminescence (EL). The monitor 13displays a setting screen for setting the recorded zone identificationinformation, camera identification information, grouping information,and various setting information. Further, the monitor 13 displays thealert notification received from the server S1.

The operation unit 14 is, for example, a user interface that detects aninput operation of a user, and is configured using a mouse, a keyboard,a touch panel, or the like. Based on the input operation of the user,the operation unit 14 receives settings of various setting types,converts the settings into signals, and outputs the signals to theprocessor 11.

The registrant database 15 includes a semiconductor memory such as a RAMand a ROM, and a storage device including any of storage devices such asa solid state drive (SSD) and a hard disk drive (HDD). The registrantdatabase 15 records the face image of each of the plurality of personsrecorded in the registrant memory table TB1.

Next, a configuration of each of the plurality of cameras C1, . . . willbe described. The camera C1 includes a communication unit 20, aprocessor 21, a memory 22, and an imaging unit 23. Note that theplurality of cameras C1, . . . have substantially the sameconfiguration. In the following description, the configuration of thecamera C1 will be described, and a description of the configuration ofanother camera C2 will be omitted.

The communication unit 20 is connected to the communication unit 30 inthe server S1 so as to be able to perform data communication via anetwork. The communication unit 20 transmits the face cut-out imagegenerated by the processor 21 and the camera identification informationof the camera C1 to the server S1.

The processor 21 as an example of an image processing unit is configuredusing, for example, a CPU or an FPGA, and performs various processes andcontrols in cooperation with the memory 22. Specifically, the processor21 refers to a program and data held in the memory 22, and executes theprogram to implement the functions of the respective units. The functionreferred to here is, for example, a function of detecting a face of aperson shown in a captured image or a function of generating a facecut-out image in which a detected face region of a person is cut out.

When the captured image captured by the imaging unit 23 is input, theprocessor 21 detects the face of the person shown in the input capturedimage. When a plurality of persons are shown in the captured image, theprocessor 21 detects a face of each of the plurality of persons. Theprocessor 21 generates the face cut-out image obtained by cutting outthe detected face of the person, and transmits the generated facecut-out image, the position coordinate information of the detected faceof each person, and the camera identification information of the cameraC1 to the server S1 via the communication unit 20.

When generating the face cut-out image, the processor 21 cuts out a faceof each of the plurality of persons in a predetermined cut-out range.When there is a region in which the cut-out range overlaps with acut-out range of a face of the other person, the processor 21 may cutout the cut-out range including the cut-out range of the face of each ofthe plurality of persons and generate a face cut-out image. In such acase, the face cut-out image is cut out by including the faces of theplurality of persons, and the position coordinate information of theface of each person included in the predetermined cut-out range isassigned to identify the detected face of each person.

The memory 22 includes, for example, a RAM as a work memory used wheneach processing of the processor 21 is executed, and a ROM that storesprograms and data defining the operation of the processor 21. The RAMtemporarily stores data or information generated or acquired by theprocessor 21. A program that defines the operation of the processor 21is written in the ROM. The memory 22 records the camera identificationinformation of the camera C1. The memory 22 may record the groupinginformation of the camera.

The imaging unit 23 includes at least a lens (not shown) and an imagesensor (not shown). The image sensor is a solid-state image capturingdevice such as a charge-coupled device (CCD) or a complementarymetal-oxide semiconductor (CMOS), and converts an optical image formedon an imaging surface into an electrical signal. The imaging unit 23outputs the captured image to the processor 21.

The server S1 respectively collates the face images of the plurality ofpersons included in the registrant memory table TB1 with the facecut-out images transmitted from the plurality of cameras C1, . . . , andexecutes the face authentication. The server S1 includes thecommunication unit 30, a processor 31, a memory 32, and the registrantdatabase 33.

The communication unit 30 is connected to the communication unit 10 inthe terminal device P1 and the communication unit 20 in the camera C1 soas to be able to perform data communication via a network. Thecommunication unit 30 transmits the set recorded zone identificationinformation, camera identification information, grouping information ofthe camera, and the registrant memory table TB1 generated by theprocessor 11 to the terminal device P1. In addition, the communicationunit 30 outputs the face cut-out image transmitted from the camera C1,the position coordinate information of the face, and the cameraidentification information to the processor 31. The communication unit30 transmits the determined result (alert notification) as the faceauthentication result generated by the processor 31 to the terminaldevice P1.

The processor 31 as an example of an image processing unit, a zonedetermination unit, a collation unit, and an output control unit isconfigured using, for example, a CPU or an FPGA, and performs variousprocesses and controls in cooperation with the memory 32. Specifically,the processor 31 refers to a program and data held in the memory 32, andexecutes the program to implement the functions of the respective units.The function referred to here is, for example, a function of generatingthe registrant memory table TB1 based on various setting information, afunction of determining a zone in which the camera is installed based onthe camera identification information, a function of, based on thedetermined zone identification information, referring to the registrantmemory table TB1 and determining whether a person shown in the facecut-out image is a person who is permitted to enter the zone, and afunction of generating an alert notification of a person who is notpermitted to enter as a determined result.

In the processor 31, for example, the face cut-out image and the cameraidentification information transmitted from the camera C1 are input fromthe communication unit 30. The processor 31 collates the cameraidentification information recorded in the memory 32 with the inputcamera identification information, and determines that the camera inwhich the face cut-out image is captured is, for example, the camera C1.The processor 31 executes the collation processing with reference to theregistrant memory table TB1 recorded in the memory 32 for each personshown in the face cut-out image transmitted from the camera C1. Inaddition, the processor 31 determines a zone in which the camera isinstalled based on the camera identification information and thegrouping information recorded in the memory 32.

Based on the determined zone identification information, the processor31 refers to the registrant memory table TB1 and determines whether theperson shown in the face cut-out image is a person (registrant) who ispermitted to enter the zone. Specifically, the processor 31 collates theface images of the persons shown in the face cut-out image with the faceimage of each of the plurality of persons recorded in the registrantmemory table TB1, respectively. The processor 31 may execute collationbased on a feature amount of the face.

When it is determined that the person shown in the face cut-out image isnot a registrant recorded in the registrant memory table TB1 as a resultof the collation, the processor 31 generates an alert notificationnotifying the user that the person shown in the face cut-out image is aperson who is not permitted to enter the zone. The processor 31superimposes a detection frame surrounding the face of a person who isnot permitted to enter on the face cut-out image based on the positioncoordinate information of the face transmitted from the camera. Theprocessor 31 transmits the alert notification including the face cut-outimage on which the detection frame is superimposed and the informationon the zone in which an image of the person is captured to the terminaldevice P1. The processor 31 may generate an alert notification includingthe face image recorded in the registrant memory table TB1 when theperson is a person recorded in the registrant memory table TB1 and isnot permitted to enter the imaged zone.

Further, when it is determined that the person shown in the face cut-outimage is a person recorded in the registrant memory table TB1 as aresult of the collation, the processor 31 further refers to theregistrant memory table TB1, and determines whether the expiration dateindicating a period for allowing the person to enter has expired.

As a determined result, when it is determined that the person shown inthe face cut-out image is the registrant recorded in the registrantmemory table TB1 and the expiration date has expired, the processor 31generates an alert notification notifying the user that the person shownin the face cut-out image is a person who is not permitted to enter thezone. Specifically, based on the position coordinate information of theface transmitted from the camera, the processor 31 superimposes thedetection frame surrounding a face region of the person whose expirationdate has expired on the face cut-out image. The processor 31 transmitsan alert notification including the face cut-out image on which thedetection frame is superimposed and the information on the zone in whichthe image of the person is captured to the terminal device P1. Theprocessor 31 may further generate an alert notification includinginformation on the expiration date and transmit the alert notificationto the terminal device P1.

The collation processing of the face by the processor 31 described abovemay be executed using learning data generated in advance. In such acase, the processor 31 may further include an AI processing unit (notshown). The AI processing unit includes, for example, a CPU, a digitalsignal processor (DSP), or an FPGA, and includes a learning modeldatabase storing feature amount data of the face images of the pluralityof persons stored in the past.

The AI processing unit (not shown) executes the collation processing ofthe face based on the face cut-out image transmitted from the camera andthe face images of the plurality of persons recorded in the registrantmemory table TB1.

The learning model database includes a semiconductor memory such as aRAM and a ROM, and a storage device including any of storage devicessuch as a solid state drive (SSD) and a hard disk drive (HDD). Thelearning model database generates or stores, for example, a program thatdefines the collation processing executed by the AI processing unit,various setting data for executing these processes, learning data usedwhen extracting and collating feature amounts of the face of the person,or the like.

Learning to generate learning data may be performed using one or morestatistical classification techniques. The statistical classificationtechniques include, for example, linear classifiers, support vectormachines, quadratic classifiers, kernel density estimation, decisiontrees, artificial neural networks, Bayesian technologies and/ornetworks, hidden Markov models, binary classifiers, multi-classclassifiers, a clustering technique, a random forest technique, alogistic regression technique, a liner regression technique, a gradientboosting technique, or the like. However, the statistical classificationtechnique used is not limited thereto.

When the server S1 acquires the captured image captured from each of theplurality of cameras C1, . . . and the identification information of thecamera that captures the captured image, the processor 31 may detect theperson shown in the captured image and generate the face cut-out image,and may also generate the position coordinate information of thedetected face for each person.

When the processor 31 determines as a determined result that a personwho is not permitted to enter the zone is shown in the face cut-outimage, the processor 31 may transmit the face cut-out image and theposition coordinate information of the face of the person who is notpermitted to enter the zone as a determined result to the terminaldevice P1. In such a case, the processor 11 in the terminal device P1generates an alert notification based on the determined result.

The memory 32 as an example of a storage includes, for example, a RAM asa work memory used when executing each processing of the processor 31,and a ROM that stores programs and data defining the operation of theprocessor 31. The RAM temporarily stores data or information generatedor acquired by the processor 31. A program that defines the operation ofthe processor 31 is written in the ROM. The memory 32 records varioussetting information and the registrant memory table TB1 transmitted fromthe terminal device P1.

The registrant database 33 includes a semiconductor memory such as a RAMand a ROM, and a storage device including any of storage devices such asan SSD or an HDD. The registrant database 33 records the face images ofthe plurality of persons recorded in the registrant memory table TB1.

FIG. 3 is a view showing an example of the registrant memory table TB1.The registrant memory table TB1 is generated and recorded by the serverS1.

The registrant memory table TB1 includes face images of face images LT1,LT2, LT3, . . . of a plurality of persons as the registrants, theinformation on a zone indicating whether or not to permit entry into thezone, and information on an expiration date as a deadline for the entry.

For example, the registrant indicated by the face image LT1 is permittedto enter the zone A and the zone B until an expiration date “2019/12/2523:59:59”. The registrant indicated by the face image LT2 is permittedto enter the zone A until an expiration date “2020/03/30 23:59:59”. Theregistrant indicated by the face image LT3 is permitted to enter thezone B until an expiration date “9999/03/30 23:59:59”.

As a result, for example, when there are a plurality of zones (areas) inthe same building, the face authentication system 100 according to thefirst embodiment can manage entrance (entry) into each zone for eachperson. In addition, the face authentication system 100 according to thefirst embodiment can easily manage entry of an employee or the like whohas a contract period, such as each of the registrants indicated by theface images LT1, LT2, into a building or zone by setting the expirationdate by the user operation, for example. Note that the employee whosecontract period is indefinite may be set to a value indicating adistance future expiration date, such as the registrant indicated by theface image LT3.

FIG. 4 is a sequence diagram showing an operation procedure example ofthe face authentication system 100 according to the first embodiment. InFIG. 4 , an example in which the person shown in the captured image isone person, the person is permitted to enter the zone and no alertnotification is generated will be described. In the operation procedureexample of the face authentication system 100 shown in FIG. 4 , anexample in which the terminal device P1 generates and records theinformation on each zone, the camera identification information and thegrouping information of the camera installed in the zone, and theregistrant memory table TB1 will be described. These information andtables may be generated and recorded by the server S1 or may begenerated and recorded by the terminal device P1.

The terminal device P1 receives settings for each zone and settings forgrouping of the cameras installed in each zone by the user operation,and records the setting information. The terminal device P1 acquires andrecords (registers) each of the face images of the plurality of personsrecorded in the registrant memory table TB1 by the user operation (St1).The terminal device P1 may receive input regarding the information onthe zone indicating whether to permit entry into the zone and theexpiration date by the user operation for each person in the acquiredface image, and generate the registrant memory table TB1.

The terminal device P1 transmits the set information on each zone,camera identification information and grouping information of the camerainstalled in each zone to the server S1 (St2).

The terminal device P1 transmits the registrant memory table TB1generated based on the user operation to the server S1 (St3).

The server S1 records (registers) the zone identification information ofeach zone, the camera identification information and groupinginformation of the camera installed in each zone that are transmittedfrom the terminal device P1, and the registrant memory table TB1 (St4).

The plurality of cameras C1, . . . capture the images of the pluralityof persons entering the zone, and detects the faces of the persons fromthe captured images, respectively. Each of the plurality of cameras C1,. . . generates the face cut-out image Pc0 obtained by cutting out aregion including the detected face of the person (St5A).

Each of the plurality of cameras C1, . . . transmits the generated facecut-out image Pc0, the position coordinate information of the face ofthe person shown in the face cut-out image Pc0, and the cameraidentification information to the server S1 (St6A).

The server S1 collates the received camera identification informationwith the registered camera identification information and groupinginformation, and determines a zone in which the face cut-out image Pc0is captured (St7A).

The server S1 collates the face image LT1 recorded in the registrantmemory table TB1 with the face cut-out image Pc0 in the zone determinedin step St7A, and determines whether the person (person in the faceimage LT1) shown in the face cut-out image Pc0 is permitted to enter(St8A).

When it is determined that the entry of the person shown in the facecut-out image Pc0 is permitted as a result of the processing in stepSt8A, the server S1 does not transmit the determined result to theterminal device P1. When it is determined that the entry of the personshown in the face cut-out image is not permitted as a result of theprocessing in step St8A, the server S1 generates an alert notificationand transmits the alert notification to the terminal device P1.

FIG. 5 is a sequence diagram showing an operation procedure example ofthe face authentication system 100 according to the first embodiment. InFIG. 5 , an example in which each face of a plurality of persons isshown in one face cut-out image, one of the plurality of persons is notpermitted to enter the zone, and an alert notification is generated willbe described. The processings in steps St1 to St4 in the operationprocedure of the face authentication system 100 shown in FIG. 5 are thesame as the processings described in the operation procedure of the faceauthentication system 100 shown in FIG. 4 , and a description thereofwill be omitted. In the operation procedure example of the faceauthentication system 100 shown in FIG. 5 , similarly to the exampleshown in FIG. 4 , an example in which the terminal device P1 generatesand records the information on each zone, the camera identificationinformation and the grouping information of the camera installed in eachzone, and the registrant memory table TB1 will be described.

The plurality of cameras C1, . . . capture the images of the pluralityof persons entering the zone, and detects the faces of the plurality ofpersons from the captured images, respectively. Each of the plurality ofcameras C1, . . . generates a face cut-out image Pc3 obtained by cuttingout a region including the face of the person detected from the capturedimage (St5B). In addition, when it is determined that the two personsare shown in the face cut-out image Pc3, each of the plurality ofcameras C1, . . . may detect the two persons, and may generate aplurality of face cut-out images Pc311, Pc312 and the positioncoordinate information of the faces of the two persons.

Each of the plurality of cameras C1, . . . transmits the generated facecut-out image Pc3, the position coordinate information of the face ofthe person shown in the face cut-out image Pc3, and the cameraidentification information to the server S1 (St6B). Each of theplurality of cameras C1, . . . may transmit a plurality of face cut-outimages Pc311, Pc312, the position coordinate information of the faces ofthe two persons and the camera identification information to the serverS1.

The server S1 collates the received camera identification informationwith the registered camera identification information and groupinginformation, and determines a zone in which the face cut-out image Pc3is captured (St7B).

In the zone determined in step St7B, the server S1 collates the faceimage recorded in the registrant memory table TB1 with the face image ofeach of the plurality of persons included in the face cut-out image Pc3,and determines whether the entry of each of the plurality of personsshown in the plurality of face cut-out images Pc3 is permitted (St8B).When the server S1 receives each of the plurality of face cut-out imagesPc311, Pc312 from the plurality of cameras C1, . . . , the positioncoordinate information of the faces of two persons, and the cameraidentification information, the server S1 may collate the face imagerecorded in the registrant memory table TB1 with each of the pluralityof face cut-out images Pc311, Pc312.

As a result of the processing in step St8B, the server S1 determinesthat one of the two persons shown in the face cut-out image Pc3 is aperson in the registered face image LT2 (person shown in the facecut-out image Pc312) and is permitted to enter the zone. Further, theserver S1 determines that the entry of the other person (the personshown in the face cut-out image Pc311) of the two persons shown in theface cut-out image Pc3 is not permitted. Based on the determined result,the server S1 superimposes a detection frame Ar31 for indicating aperson who is not permitted to enter the zone and is not located on theface cut-out image Pc3 using the position coordinate information of theface of the other person (the person cut out as the face cut-out imagePc311) in the face cut-out image Pc3 transmitted from each of theplurality of cameras C1, . . . . The server S1 generates an alertnotification Ar3 including the face cut-out image Pc3 and transmits thealert notification Ar3 to the terminal device P1 (St9B).

The terminal device P1 displays the alert notification Ar3 transmittedfrom the server S1 on the monitor 13 (St10B).

Since the face authentication system 100 according to the firstembodiment does not display the determined result of each of theplurality of persons who is permitted to enter the zone on the monitor13, it is possible to reduce a work load of a management work of theuser. Therefore, the user can concentrate on only monitoring of personswho are not permitted to enter a zone, which is a purpose of themanagement task, and can perform the work.

The face authentication system 100 according to the first embodiment canmore clearly show a person who is not permitted to enter the zone evenwhen the face cut-out image includes a plurality of persons bysuperimposing the detection frame surrounding the face of the person whois not permitted to enter the zone. As a result, the face authenticationsystem 100 can detect only a person who is not permitted to enter thezone even when another person who is impersonating a person who ispermitted to enter the zone enters the zone or when a person who is notpermitted to enter the zone has entered the zone among the plurality ofpersons who are permitted to enter the zone, for example. Therefore, theuser can easily grasp the person who is not permitted to enter the zonefrom among the plurality of persons.

FIG. 6A is a view illustrating an example of an alert notification Ar1.The alert notification shown in FIG. 6A is generated by the server S1when a person who is recorded in the registrant memory table TB1 andwhose expiration date has expired enters the zone. The terminal deviceP1 displays the alert notification Ar1 transmitted from the server S1 onthe monitor 13.

The alert notification includes the information on a zone in which animage of a person who is not permitted to enter the zone is captured,and a face cut-out image Pc1 in which an image of a person who is notpermitted to enter the zone is captured.

The alert notification Ar1 is generated by the server S1 when it isdetermined in step St9 in the operation procedure example of the faceauthentication system 100 shown in FIG. 5 that there is a person whoseexpiration date has expired among the plurality of persons shown in theface cut-out image Pc1. The alert notification Ar1 is transmitted fromthe server S1 to the terminal device P1 and displayed on the monitor 13.The alert notification Ar1 includes the face cut-out image Pc1 on whicha detection frame Ar11 indicating a person who is not permitted to entera zone as a detection image is superimposed. In addition, when theexpiration date of the person who is not permitted to enter the zone hasexpired, the alert notification Ar1 does not include a registered imageregardless of whether the face image of the person is recorded(registered) in the registrant memory table TB1, as shown in FIG. 6A.

FIG. 6B is a view illustrating an example of an alert notification Ar2.The alert notification shown in FIG. 6B is generated by the server S1when a person enters a zone other than the zone recorded in theregistrant memory table TB1 (that is, the zone where the entry ispermitted). The terminal device P1 displays the alert notification Ar2transmitted from the server S1 on the monitor 13.

The alert notification Ar2 is generated by the server S1 when it isdetermined in step St9 in the operation procedure example of the faceauthentication system 100 shown in FIG. 5 that a person recorded in theregistrant memory table TB1, among the plurality of persons shown in aface cut-out image Pc2, has entered an unpermitted zone. The alertnotification Ar2 is transmitted from the server S1 to the terminaldevice P1 and displayed on the monitor 13. The alert notification Ar2includes the face cut-out image Pc2 in which a detection frame Ar21indicating a person who is not permitted to enter a zone as a detectionimage is superimposed. The alert notification Ar2 is generated byincluding, as a registered image Rc2, a face image recorded (registered)in the registrant memory table TB1 when a person is a person recorded(registered) in the registrant memory table TB1 and enters a zone otherthan a permitted zone as shown in FIG. 6 .

FIG. 6C is a view illustrating an example of the alert notification Ar3.The alert notification shown in FIG. 6C is generated by the server S1when a person who is not recorded in the registrant memory table TB1enters the zone. The terminal device P1 displays the alert notificationAr3 transmitted from the server S1 on the monitor 13.

The alert notification Ar3 is generated by the server S1 when it isdetermined in step St9 in the operation procedure example of the faceauthentication system 100 shown in FIG. 5 that a person who is not beenrecorded (registered) in the registrant memory table TB1, among theplurality of persons shown in the face cut-out image Pc3, has enteredthe zone. The alert notification Ar3 is transmitted from the server S1to the terminal device P1 and displayed on the monitor 13. The alertnotification Ar3 includes the face cut-out image Pc3 in which adetection frame Ar31 indicating a person who is not permitted to enterthe zone as a detection image is superimposed. In addition, the alertnotification Ar3 does not include a registered image because the faceimage of the person who is not permitted to enter the zone is notrecorded (registered), as shown in FIG. 6C.

As described above, the face authentication system 100 according to thefirst embodiment includes a storage (memory 12 in the terminal device P1or memory 32 in the server S1) that stores a face image of each of aplurality of persons who are permitted to enter in advance andinformation on a zone in association with each other, an imageprocessing unit (processor 21 in the camera or processor 31 in theserver S1) that detects faces of one or more persons from an imagecaptured by at least one camera holding identification information, andgenerates a face cut-out image obtained by cutting out a detected faceportion, a zone determination unit (processor 31 in the server S1) thatdetermines a zone in which an image of a person in the face cut-outimage is captured based on the identification information of the camera,a collation unit (processor 31 in the server S1) that collates the facecut-out image with each of the face images of the plurality of personswho are permitted to enter the zone in advance, and an output controlunit (processor 11 in the terminal device P1 or processor 31 in theserver S1) that superimposes and outputs a detection frame indicating aface of a person who is not permitted to enter the zone in the facecut-out image when it is determined that there is not the face of theperson shown in the face cut-out image among the face images of theplurality of persons who are permitted to enter the zone in advance.

As a result, the face authentication system 100 according to the firstembodiment detects and outputs a person who is not permitted to enteramong the plurality of persons in each zone as the monitored area. Thatis, since the face authentication system 100 can transmit an alertnotification only when a person who is not permitted to enter isdetected, and can display the alert notification on the monitor 13 inthe terminal device P1 used by the user, it is possible to moreefficiently support a monitoring work of the user.

The storage (the memory 12 in the terminal device P1 or the memory 32 inthe server S1) in the face authentication system 100 according to thefirst embodiment further stores, in association with a face image ofeach of the plurality of persons, information on the expiration date forwhich the entry is permitted. The collation unit (the processor 31 inthe server S1) further determines whether the expiration datecorresponding to the face image of the person has expired when it isdetermined that there is a face of a person shown in the face cut-outimage among the face images of the plurality of persons who arepermitted to enter the zone in advance. The output control unit (theprocessor 11 in the terminal device P1 or the processor 31 in the serverS1) superimposes and outputs the detection frame indicating the face ofthe person whose expiration date has expired on the face cut-out image,when it is determined that the expiration date has expired as adetermined result. As a result, the face authentication system 100according to the first embodiment can generate the alert notification asa determined result in which a person who is not permitted to enter thezone is emphasized even when the plurality of persons are shown in thecaptured image and the plurality of persons are shown in the generatedface cut-out image. Therefore, even when each of a plurality of personsis shown in the face cut-out image, the user can easily confirm theperson who is not permitted to enter the zone.

The storage (the memory 12 in the terminal device P1 or the memory 32 inthe server S1) in the face authentication system 100 according to thefirst embodiment further stores, in association with a face image ofeach of the plurality of persons, information on the expiration date forwhich the entry is permitted. In addition, the collation unit (theprocessor 31 in the server S1) deletes the face image of the personwhose expiration date has expired, the information on the zone and theinformation on the expiration date recorded in association with the faceimage of the person. As a result, since the server S1 in the faceauthentication system 100 according to the first embodiment can deleteunnecessary data, it is possible to reduce data recording capacityrequired for recording the registrant memory table TB1.

The storage (the memory 12 in the terminal device P1 or the memory 32 inthe server S1) in the face authentication system 100 according to thefirst embodiment stores the information on the expiration date for whichthe entry is permitted in association with the face image of each of theplurality of persons, and continuously records the face image of theperson whose expiration date has expired, the information on the zoneand the information on the expiration date recorded in association withthe face image of the person. As a result, the face authenticationsystem 100 according to the first embodiment can save the trouble ofregistering the face image again when a person who has been permitted toenter the zone in the past is permitted again after the expiration dateor the like.

Although various embodiments have been described above with reference tothe accompanying drawings, the present disclosure is not limited to suchembodiment. It will be apparent to those skilled in the art that variouschanges, modifications, substitutions, additions, deletions, andequivalents can be conceived within the scope of the claims, and itshould be understood that these changes, modifications, substitutions,additions, deletions, and equivalents also belong to the technical scopeof the present disclosure. In addition, constituent elements in thevarious embodiments described above may be arbitrarily combined within arange not departing from the gist of the invention.

Each of the plurality of cameras C1, . . . in the face authenticationsystem 100 according to the first embodiment may store data of acaptured video in an external storage device (not shown). The externalstorage device is connected to the plurality of cameras C1, . . . andthe server S1 so as to be able to perform data communication. When thealert notification is included in the determined result transmitted fromthe server S1, the terminal device P1 may request the data of thecaptured video including an imaging time before and after the capturedimage in which the alert notification is generated from the externalstorage device (not shown) via the server S1. The terminal device P1 caneasily confirm the captured video before and after the alertnotification is generated by displaying the data of the captured videotransmitted from the server S1 on the monitor 13.

INDUSTRIAL APPLICABILITY

The present disclosure is useful as the face authentication system and aface authentication method that detect and output a person who is notpermitted to enter among a plurality of persons entering the monitoredarea and more efficiently support the monitoring work of the user.

The present application is based upon Japanese Patent Application(Patent Application No. 2020-003752 filed on Jan. 14, 2020), the contentof which is incorporated herein by reference.

What is claimed is:
 1. A face authentication system comprising: a memory configured to store, in advance, a face image of each of a plurality of persons that is permitted to enter a zone in association with information on the zone; a processor configured to detect faces of a plurality of people in an image captured by a camera having identification information, and generate face cut-out images obtained by cutting out detected face portions of the plurality of people; the processor configured to determine a zone where the image is captured based on the identification information of the camera; the processor configured to collate the face cut-out images with the face image of each of the plurality of persons that is permitted to enter the zone; the processor configured to display the face cut-out images of the plurality of people; and the processor configured to, when the processor determines by collation that a face image of a first one of the plurality of people appearing in the face cut-out images is not among the face image of each of the plurality of persons that is permitted to enter the zone, superimpose a detection frame indicating the face image of the first one of the plurality of people that is not permitted to enter the zone on one of the face cut-out images corresponding to the first one of the plurality of people, wherein, when the processor determines by collation that a face image of a second one of the plurality of people appearing in the face cut-out images is among the face image of each of the plurality of persons that is permitted to enter the zone, the processor does not superimpose the detection frame on one of the face cut-out images corresponding to the second one of the plurality of people.
 2. The face authentication system according to claim 1, wherein the memory further stores, in association with the face image of each of the plurality of persons, information on an expiration date for which entry in the zone is permitted; wherein the processor further determines whether the expiration date corresponding to the face image of the second one of the plurality of people, that is permitted to enter the zone, has expired when the processor determines by the collation that the second one of the plurality of people is permitted to enter the zone; and wherein the processor superimposes the detection frame indicating the face image of the second one of the plurality of people whose expiration date has expired on a second one of the face cut-out images corresponding to the second one of the plurality of people when the processor determines that the expiration date has expired.
 3. The face authentication system according to claim 1, wherein the memory further stores, in association with the face image of each of the plurality of persons, information on an expiration date for which entry in the zone is permitted; and wherein the processor deletes, from the memory, a face image of a person whose expiration date has expired, and deletes the information on the zone and the information on the expiration date which are stored in association with the face image of the person whose expiration date has expired.
 4. The face authentication system according to claim 1, wherein the memory stores information on an expiration date for which entry in the zone is permitted in association with the face image of each of the plurality of persons, and continuously stores a face image of a person whose expiration date has expired, and continuously stores the information on the zone and the information on the expiration date which are stored in association with the face image of the person whose expiration date has expired.
 5. A face authentication method comprising: storing, in advance, a face image of each of a plurality of persons that is permitted to enter a zone in association with information on the zone; detecting faces of a plurality of people in an image captured by a camera having identification information, and generating face cut-out images obtained by cutting out detected face portions of the plurality of people; determining a zone where the image is captured based on the identification information of the camera; collating, by a processor, the face cut-out images with the face image of each of the plurality of persons that is permitted to enter the zone; displaying, by the processor, the face cut-out images of the plurality of people; and when the processor determines by collation that a face image of a first one of the plurality of people appearing in the face cut-out images is not among the face image of each of the plurality of persons that is permitted to enter the zone, superimposing a detection frame indicating the face image of the first one of the plurality of people that is not permitted to enter the zone on one of the face cut-out images corresponding to the first one of the plurality of people, wherein, when the processor determines by collation that a face image of a second one of the plurality of people appearing in the face cut-out images is among the face image of each of the plurality of persons that is permitted to enter the zone, the processor does not superimpose the detection frame on one of the face cut-out images corresponding to the second one of the plurality of people.
 6. The face authentication system according to claim 1, wherein the memory stores, in advance, a face image of each of a second plurality of persons that is permitted to enter a second zone in association with information on the second zone, the processor is configured to collate the one of the face cut-out images of the first one of the plurality of people that is not permitted to enter the zone with the face image of each of the second plurality of persons that is permitted to enter the second zone, and the processor is configured to, when the processor determines by collation that the first one of the plurality of people that is not permitted to enter the zone is permitted to enter the second zone, output the one of the face cut-out images with the detection frame together with the face image of one of the second plurality of persons which indicates that the first one of the plurality of people is permitted to enter the second zone, whereby the one of the face cut-out images with the detection frame is different than the face image of the one of the second plurality of persons which indicates that the first one of the plurality of people is permitted to enter the second zone.
 7. The face authentication system according to claim 1, wherein the memory stores, in advance, face images a second plurality of persons that is permitted to enter a plurality of zones in association with information on the plurality of zones, the zone being one of the plurality of zones, the processor is configured to collate the one of the face cut-out images of the first one of the plurality of people that is not permitted to enter the zone with the face images of the second plurality of persons that is permitted to enter the plurality of zones, and the processor is configured to, when the processor determines by collation that the first one of the plurality of people that is not permitted to enter the zone is not permitted to enter any of the plurality of zones, output the one of the face cut-out images with the detection frame together with indicia which indicates that the first one of the plurality of people is not permitted to enter any of the plurality of zones.
 8. The face authentication system according to claim 1, wherein the processor, when the processor determines by collation that there is the face image of the second one of the plurality of people appearing in the face cut-out images among the face image of each of the plurality of persons that is permitted to enter the zone, does not output a determination result.
 9. The face authentication system according to claim 1, wherein the processor is configured to display, along with the face cut-out images of the plurality of people, an area for displaying a registered image which is stored in the memory in advance.
 10. The face authentication system according to claim 9, wherein the processor is configured to display, in the area for displaying the registered image, indicia which indicates that the first one of the plurality of people is not permitted to enter the zone when the processor determines by the collation that the face image of the first one of the plurality of people appearing in the face cut-out images is not among the face image of each of the plurality of persons that is permitted to enter the zone.
 11. The face authentication system according to claim 10, wherein the processor is configured to display, along with the face cut-out images of the plurality of people and the indicia, an alert notification which indicates that one of the plurality of people appearing in the face cut-out images is not permitted to enter the zone.
 12. The face authentication system according to claim 11, wherein the alert notification includes information on the zone where the image is captured.
 13. The face authentication system according to claim 12, wherein the area for displaying the registered image and the alert notification do not include any face image of the plurality of persons, which is stored in the memory in advance, when the processor determines by the collation that the first one of the plurality of people is not permitted to enter the zone and the processor determines, by the collation, that the face image of the first one of the plurality of people is not among the face image of each of the plurality of persons that is stored in the memory in advance.
 14. The face authentication system according to claim 12, wherein the area for displaying the registered image and the alert notification do not include any face image of the plurality of persons, which is stored in the memory in advance, when the processor determines by the collation that the first one of the plurality of people is not permitted to enter the zone, regardless of whether a face image of the first one of the plurality of people is stored in the memory in advance.
 15. A face authentication system comprising: a memory configured to store, in advance, a face image of each of a plurality of persons that is permitted to enter a zone in association with information on the zone; a processor configured to detect faces of a plurality of people in an image captured by a camera having identification information, and generate face cut-out images obtained by cutting out detected face portions of the plurality of people; the processor configured to determine a zone where the image is captured based on the identification information of the camera; the processor configured to collate the face cut-out images with the face image of each of the plurality of persons that is permitted to enter the zone; the processor configured to display the face cut-out images of the plurality of persons; and the processor configured to, when the processor determines by collation that: a face image of a second one of the plurality of people appearing in the face cut-out images is not among the face image of each of the plurality of persons that is permitted to enter the zone where the image is capture; and the face image of the second one of the plurality of people is among the face image of each of the plurality of persons that is permitted to enter a different zone than the zone where the image is capture, superimpose a detection frame indicating the face image of the second one of the plurality of people that is not permitted to enter the zone where the image is capture on one of the face cut-out images corresponding to the second one of the plurality of people, wherein, when the processor determines by collation that a face image of a first one of the plurality of people appearing in the face cut-out images is among the face image of each of the plurality of persons that is permitted to enter the zone where the image is captured, the processor does not superimpose the detection frame on one of the face cut-out images corresponding to the first one of the plurality of people.
 16. The face authentication system according to claim 15, wherein the processor is configured to display, along with the face cut-out images of the plurality of people, an area for displaying a registered image which is stored in the memory in advance.
 17. The face authentication system according to claim 16, wherein the processor is configured to display, in the area for displaying the registered image, the face image of one of the plurality of persons which is stored in the memory in advance and which indicates that the second one of the plurality of people is permitted to enter the different zone than the zone where the image is capture.
 18. The face authentication system according to claim 17, wherein the processor is configured to display, along with the face cut-out images of the plurality of people and the face image of the one of the plurality of persons which is stored in the memory in advance and which indicates that the second one of the plurality of people is permitted to enter the different zone, an alert notification which indicates that one of the plurality of people appearing in the face cut-out images is not permitted to enter the zone where the image is captured.
 19. The face authentication system according to claim 18, wherein the alert notification includes information on the zone where the image is captured.
 20. The face authentication system according to claim 19, wherein the face image of the second one of the plurality of people is different than the face image of the one of the plurality of persons which is stored in the memory in advance and which indicates that the second one of the plurality of people is permitted to enter the different zone. 